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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 



Confirmation No.: 9217 
Art Unit: 2624 

BACKGROUND ESTIMATION AND Examiner: I. I. Sherali 

SEGMENTATION BASED ON RANGE AND 
COLOR 



AMENDMENT IN RESPONSE TO RESTRICTION REQUIREMENT 



In re Patent Application of: 
Gaile Gordon et al. 

Application No.: 10/703,393 

Filed: November 7, 2003 

For: 



MS Amendment 
Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 

Sir: 



INTRODUCTORY COMMENTS 



Docket No.: 34528801 3US 
(PATENT) 



In response to the Office Action dated April 23, 2007, please amend the above- 
identified U.S. patent application as follows: 

Amendments to the Claims are reflected in the listing of claims which begins on 
page 2 of this paper. 

Remarks/Arguments begin on page 7 of this paper. 
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AMENDMENTS TO THE CLAIMS 

1 . (Original) A method for distinguishing between background and foreground objects 
in an image, comprising the steps of: 

i) determining respective range values for pixels in each of a plurality of related 
images; 

ii) determining respective intensity values for pixels in each of said plurality of 
related images; 

iii) establishing a multi-dimensional background model for at least some of said 
pixels on the basis of said range values and said intensity values; 

iv) comparing the range value for a particular pixel in a given image with the 
background model for that pixel, and labeling the pixel as a foreground pixel if the range 
value differs from the background model by a range threshold; and 

v) comparing the intensity value for said particular pixel in a given image with the 
background model for that pixel, and labeling the pixel as a foreground pixel if the intensity 
value differs from the background model by an intensity threshold. 

2. (Original) The method of claim 1 further including the step of determining whether 
the range value for said particular pixel is valid, and performing said comparing step (iv) 
only if said range value is valid. 
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3. (Original) The method of claim 2 wherein determining whether the range value for 
said particular pixel is valid further includes: 

computing a confidence value; and 

comparing said confidence value to a threshold to determine if the range value for 
said particular pixel is valid. 

4. (Original) The method of claim 2 further including the step of determining whether 
background range data for said particular pixel is valid, and if said background range data 
is not valid, labeling the particular pixel as a foreground pixel in response to the comparing 
step (iv) for neighboring pixels if a range gradient associated with said particular pixel is 
less than a gradient threshold. 

5. (Original) The method of claim 1 further including the step of determining whether 
the luminance of said particular pixel is greater than a designated minimum value, and 
performing said comparing step (v) only if the luminance is greater than said minimum 
value . 

6. (Original) The method of claim 1 further including the step of determining the ratio 
of the intensity value for said particular pixel to the background intensity value for that 
pixel, and labeling the pixel as a foreground pixel if said ratio is less than a first 
predetermined value. 

7. (Original) The method of claim 6 wherein said first predetermined value is based 
upon a decrease in luminance due to shadows in an image. 

8. (Original) The method of claim 6 further including the step of labeling the pixel as a 
foreground pixel if said ratio is greater than a second predetermined value. 
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9. (Original) The method of claim 8 wherein said second predetermined value is 
based upon an increase in luminance due to interreflections in an image. 

10. (Original) The method of claim 1 further including the step of alternatively labeling 
the pixel as a foreground pixel if the luminance of the pixel is greater than a designated 
minimum value by a predetermined factor. 

1 1 . (Original) The method of claim 10 wherein said factor is at least 2. 

12. (Original) The method of claim 10 wherein said alternative labeling step is 
performed only if the luminance of the background texture model for the pixel is less than 
said designated minimum value. 

13. (Original) The method of claim 1 further including the step of updating said 
background model on the basis of range values and intensity values obtained from a 
sequence of N images. 

14. (Original) The method of claim 13 further including the steps of detecting image-to- 
image changes in said range values and intensity values, and effectively adjusting the 
value of N in accordance with the detected changes. 

1 5. (Original) The method of claim 14 wherein the value of N is effectively increased 
when the magnitude of detected changes increases. 

16. (Original) The method of claim 1 wherein said intensity values are one-dimensional 
values corresponding to the luminance of pixels. 

17. (Original) The method of claim 1 wherein said intensity values are two-dimensional 
values which are invariant to luminance. 
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1 8. (Original) The method of claim 1 wherein said intensity values represent a three- 
dimensional color space. 

19. (Original) The method of claim 1 8 further including the step of normalizing said 
intensity values according to luminance. 

20. (Original) A system for identifying foreground objects in an image comprising: 
a sensor which determines intensity values for pixels within an image; 

a range processor which determines distance values for pixels within said image; 

a background estimator which computes a multi-dimensional background model for 
said pixels on the basis of said intensity values and said distance values; and 

a discriminator which compares intensity and distance values for pixels in an image 
to said background model and determines whether said pixels represent a foreground 
object. 

21 . (Original) The system of claim 20 wherein said background estimator generates a 
multi-dimensional histogram of intensity and distance values for a pixel within a plurality of 
related images, and computes a background model for said pixel by means of a clustering 
technique. 

22. (Original) The system of claim 20 wherein said discriminator determines the 
differences between the intensity and distance values for a pixel in an image and the 
background model for that pixel, and compares said differences to respective intensity and 
distance thresholds. 
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23. (Original) The system of claim 22 wherein said discriminator selectively adjusts said 
intensity threshold in accordance with the result of a comparison based upon said distance 
threshold. 

24. (Original) The system of claim 22 wherein said discriminator increases said 
intensity threshold if the difference between the distance value for a pixel in an image and 
the background model for that pixel is less than said distance threshold. 

25. (Original) The system of claim 20 wherein said intensity values are one- 
dimensional values corresponding to the luminance of pixels. 

26. (Original) The system of claim 20 wherein said intensity values are two-dimensional 
values which are invariant to luminance. 

27. (Original) The system of claim 20 wherein said intensity values represent a three- 
dimensional color space. 

28. (Original) The system of claim 20 further including an intensity processor which 
normalizes said intensity values according to luminance. 

29. -33. (Cancelled) 



34528-801 3 .US00/LEGAL1 3 1 94486. 1 



6 



Application No. 10/703,393 



Docket No.: 34528801 3US 



REMARKS 

In the above-referenced Office Action, the Examiner divided the claims into the 
following groups: 

I. Claims 1-28, drawn to a method and a system for distinguishing between 
background and foreground objects in an image; and 

II. Claims 29-33, drawn to a method and a system for determining background 
model using multi-dimension histogram. 

In response, the applicants elect Group I without traverse. Non-elected claims 29- 
33 have been cancelled. 

Accordingly, the applicants have elected to begin prosecution with examination of 
claims 1-28. 

The Commissioner is hereby authorized and requested to charge any deficiency in 
fees or credit any overpayment herein to Deposit Account No. 50-0665. 



Dated: May lO ,2007 




Registration No.: 34,807 
PERKINS COIE LLP 
P.O. Box 1247 

Seattle, Washington 981 1 1 -1 247 
(206) 359-8000 
(206) 359-7198 (Fax) 
Attorney for Applicant 
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